Mr-ecocd: an Edge Clustering Algorithm for Overlapping Community Detection on Large-scale Network Using Mapreduce

Overlapping community detection is progressively becoming an important issue in complex networks. Many in-memory overlapping community detection algorithms have been proposed for graphs with thousands of nodes. However, analyzing massive graphs with millions of nodes is impossible for the traditional algorithm. In this paper, we propose MR-ECOCD, a novel distributed computation algorithm using MapReduce to detect overlapping communities efficiently on large-scale network. Firstly, the similarities of all adjacent edges are calculated by SimilarityMap algorithm to measure the distance of edges. Secondly, we define the direct edge communities (DEC) and mergeable direct edge communities (MDEC) based on edge density clustering method. Then, MarkMap algorithm and ClusteringReduce algorithm are designed to mark DEC and merge MDEC respectively for getting finally edge communities (FEC). Finally, we transform the FEC into node communities, and a node is an overlapping node in node communities if it belongs to different edges in different FEC. MR-ECOCD consists of four major stages, and all operations are executed in parallel using MapReduce. Extensive experiments show that our algorithm can effectively and fast detect overlapping communities.